Recall-Oriented Understudy For Gisting Evaluation Research Articles

Bidirectional recommendation in HR analytics through text summarization

For over a decade, online job portals have been providing their services to both job seekers and employers in search of hiring opportunities. Because of the high demand for recruitment, it is insufficient to use conventional hiring methods to find a suitable candidate to fill the position. Validating resumes online is challenging due to the potential for manual errors, making the process inherently risky. The bidirectional method comprises named entity recognition (NER) for extracting the required resumes for recruiters. Cosine similarity shows the match percentage of resumes for the job requirements and vice versa. In an attempt to tackle an issue of unregistered words, a solution called decoder attention with pointer network (DA-PN) has been introduced. This method incorporates the use of coverage mechanism to prevent word repetition through generated text summary. DA-PN+Cover method with mixed learning objective (MLO) (DA-PN+Cover+MLO) is utilized for protecting grow of increasing faults in generated text summary. Performance of proposed method is estimated using evaluation indicator recall oriented understudy for gisting evaluation (ROUGE) and attains an average of 27.47 which is comparatively higher than existing methods.

Video Transcripts Summarization using OpenAI Whisper and GPT Model

Abstract: In today’s digital age, a vast amount of video content is generated and shared on the internet every minute. However, extracting relevant information from these videos can be time-consuming and challenging. This is where video transcript summarization comes in, providing a concise summary of video content without the need to watch the entire video. The video transcript summarization system aims to streamline the process of extracting key insights and information from video content by generating concise and informative summaries from their transcripts. In the dynamic landscape of video content, existing approaches to transcript summarization have encountered challenges that limit their effectiveness. Issues such as compromised accuracy, prolonged processing times and a restricted focus on YouTube's captioned videos have prompted the exploration of alternative solutions. This paper introduces a novel approach that tackles these limitations head-on, employing innovative technologies to redefine the landscape of video transcript summarization. To address the identified limitations, we propose an alternative strategy that hinges on the utilization of advanced technologies like OpenAI Whisper Model Automatic Speech Recognition(ASR) for video’s Transcription and Generative Pre-trained Transformer (GPT) model for Summarization. Through the integration of the OpenAI Whisper model for transcription and GPT models for summarization, our proposed approach strives to redefine standards in accuracy, processing efficiency, multilingual support, and exhibit the capability to generate either extractive or abstractive summaries. In the context of natural language processing (NLP) and text summarization ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics will be used to evaluate the quality of generated summaries compared to human referenced summaries

Exploring the potential of data augmentation in poetry generation with small-scale corpora

Poetry generation is a complex task in the field of natural language processing, especially when working with small datasets. Data augmentation techniques have been shown to be an effective way to improve the performance of deep learning models in various tasks, including image classification and speech recognition. Therefore, this study focuses on exploring the impact of four different data augmentation methods - Synonym Replacement, Random Insertion, Random Swap, and Random Deletion - on the performance of poetry generation with a small poetry dataset. The results of the study reveal that Random Insertion performed well in terms of Bilingual Evaluation Understudy (BLEU), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), and manual evaluation when compared to other data augmentation techniques. Synonym Replacement performed poorly in all three evaluations. This study confirms the potential value of data augmentation technology in poetry generation tasks and provides innovative perspectives and directions for future research in this area. Data augmentation can be employed to help address the problem of limited data in poetry generation tasks and enhance the efficiency of deep learning models. Future research could focus on exploring more advanced data augmentation techniques and their impact on poetry generation tasks.

An Abstractive Text Summarization using Decoder Attention with Pointer Network

In contemporary times, an abundance of unstructured data prevails across social media and the web. Text summarization, a process aimed at distilling relevant information concisely without altering its core meaning, has become crucial. Manual text summarization is resource-intensive, prompting the exploration of automated methods. While deep learning algorithms, particularly in abstractive text summarization, have gained popularity, further research is needed to understand their integration with semantic-based or structure-based approaches. This research leverages a dataset of 1,735 resumes sourced from Kaggle to propose a novel framework. The framework combines semantic data transformations and deep learning approaches to enhance abstractive text summarization. A key focus is addressing the challenge of handling unregistered words. The proposed solution, Decoder Attention with Pointer Network (DA-PN), is introduced. DA-PN incorporates a coverage mechanism to mitigate word repetition in generated text summaries, thereby improving the quality of summaries. The method aims to safeguard against the propagation of errors in generated text summaries. The performance of the proposed approach is evaluated using the Recall Oriented Understudy for Gisting Evaluation (ROUGE) indicator. Notably, the proposed method achieves an average ROUGE score of 26.28, surpassing existing methods. The emphasis on combining semantic data transformations, deep learning, and addressing specific challenges like word repetition sets this research apart in the field of abstractive text summarization.

Vision-Language Model for Generating Textual Descriptions From Clinical Images: Model Development and Validation Study.

The automatic generation of radiology reports, which seeks to create a free-text description from a clinical radiograph, is emerging as a pivotal intersection between clinical medicine and artificial intelligence. Leveraging natural language processing technologies can accelerate report creation, enhancing health care quality and standardization. However, most existing studies have not yet fully tapped into the combined potential of advanced language and vision models. The purpose of this study was to explore the integration of pretrained vision-language models into radiology report generation. This would enable the vision-language model to automatically convert clinical images into high-quality textual reports. In our research, we introduced a radiology report generation model named ClinicalBLIP, building upon the foundational InstructBLIP model and refining it using clinical image-to-text data sets. A multistage fine-tuning approach via low-rank adaptation was proposed to deepen the semantic comprehension of the visual encoder and the large language model for clinical imagery. Furthermore, prior knowledge was integrated through prompt learning to enhance the precision of the reports generated. Experiments were conducted on both the IU X-RAY and MIMIC-CXR data sets, with ClinicalBLIP compared to several leading methods. Experimental results revealed that ClinicalBLIP obtained superior scores of 0.570/0.365 and 0.534/0.313 on the IU X-RAY/MIMIC-CXR test sets for the Metric for Evaluation of Translation with Explicit Ordering (METEOR) and the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) evaluations, respectively. This performance notably surpasses that of existing state-of-the-art methods. Further evaluations confirmed the effectiveness of the multistage fine-tuning and the integration of prior information, leading to substantial improvements. The proposed ClinicalBLIP model demonstrated robustness and effectiveness in enhancing clinical radiology report generation, suggesting significant promise for real-world clinical applications.

Deep sequential pattern mining for readability enhancement of Indonesian summarization

In text summarization research, readability is a great issue that must be addressed. Our hypothesis is readability can be accomplished by using text representations that keep the meaning of text documents intact. Therefore, this study aims to combine sequential pattern mining (SPM) in producing a sequence of a word as text representation with unsupervised deep learning to produce an Indonesian text summary called DeepSPM. This research uses PrefixSpan as an SPM algorithm and deep belief network (DBN) as an unsupervised deep learning method. This research uses 18,774 Indonesian news text from IndoSum. The readability aspect is evaluated by recall-oriented understudy for gisting evaluation (ROUGE) as a co-selection-based analysis; Dwiyanto Djoko Pranowo metrics, Gunning fog index (GFI), and Flesch-Kincaid grade level (FKGL) as content-based analysis; and human readability evaluation with two experts. The experiment result shows that DeepSPM yields better than DBN, with the F-measure value of ROUGE-1 enhanced to 0.462, ROUGE-2 is 0.37, and ROUGE-L is 0.41. The significance of ROUGE results also be tested using T-Test. The content-based analysis and human readability evaluation findings are conformable with the findings of co-selection-based analysis that generated summaries are only partially readable or have a medium level of readability aspect.

Ar-CM-ViMETA: Arabic Image Captioning based on Concept Model and Vision-based Multi-Encoder Transformer Architecture

Image captioning is a major artificial intelligence research field that involves visual interpretation and linguistic description of a corresponding image. Successful image captioning relies on acquiring as much information as feasible from the original image. One of these essential bits of knowledge is the topic or the concept that the image is associated with. Recently, concept modeling technique has been utilized in English image captioning for completely capturing the image contexts and make use of these contexts to produce more accurate image descriptions. In this paper, a concept-based model is proposed for Arabic Image Captioning (AIC). A novel Vision-based Multi-Encoder Transformer Architecture (ViMETA) is proposed for handling the multi-outputs result from the concept modeling technique while producing the image caption. BiLingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) standard metrics have been used to evaluate the proposed model using the Flickr8K dataset with Arabic captions. Furthermore, qualitative analysis has been conducted to compare the produced captions of the proposed model with the ground truth descriptions. Based on the experimental results, the proposed model outperformed the related works both quantitatively, using BLEU and ROUGE metrics, and qualitatively

Fully automatic summarization of radiology reports using natural language processing with large language models

PurposeNatural language processing using language models has yielded promising results in various fields. Language models can help improve the workflow of radiologists. This retrospective study aimed to construct and evaluate language models for automatic summarization of radiology reports. MethodsTwo radiology report datasets from the MIMIC Chest X-ray (MIMIC-CXR) database and the Japan Medical Image Database (JMID) were included in this study. The MIMIC-CXR is an open database comprising chest radiograph reports. The JMID is a large database comprising computed tomography and magnetic resonance imaging reports from 10 academic medical centers in Japan. A total of 128,032 and 1,101,271 reports were included in this study from the MIMIC-CXR database and JMID, respectively. Four Text-to-Text Transfer Transformer (T5) models were constructed. Recall-Oriented Understudy for Gisting Evaluation (ROUGE), a quantitative metric, was used to evaluate the quality of the text summarized from 19,205 and 58,043 test sets from the MIMIC-CXR and JMID, respectively. The Wilcoxon signed-rank test was used to evaluate the differences among the ROUGE values of the four T5 models. Moreover, the subsets of automatically summarized text in the test sets were manually evaluated by two radiologists. The best T5 models were selected for automatic summarization using the Wilcoxon signed-rank test. ResultsThe quantitative metrics of the best T5 models were as follows: ROUGE-1 = 57.75 ± 30.99, ROUGE-2 = 49.96 ± 35.36, and ROUGE-L = 54.07 ± 32.48 in the MIMIC-CXR; and ROUGE-1 = 50.00 ± 29.24, ROUGE-2 = 39.66 ± 30.21, and ROUGE-L = 47.87 ± 29.44 in the JMID. The radiologists’ evaluations revealed 86% and 85% of the texts automatically summarized from the MIMIC-CXR and JMID, respectively, to be clinically useful. ConclusionThe T5 models constructed in this study were able to perform automatic summarization of the radiology reports. The radiologists’ evaluations demonstrated most of the automatically summarized texts to be clinically valuable.

Automatic Update Summarization by a Multiobjective Number-One-Selection Genetic Approach.

Currently, the explosive growth of the information available on the Internet makes automatic text summarization systems increasingly important. A particularly relevant challenge is the update summarization task. Update summarization differs from traditional summarization in its dynamic nature. While traditional summarization is static, that is, the document collections about a specific topic remain unchanged, update summarization addresses dynamic document collections based on a specific topic. Therefore, update summarization consists of summarizing the new document collection under the assumption that the user has already read a previous summarization and only the new information is interesting. The multiobjective number-one-selection genetic algorithm (MONOGA) has been designed and implemented to address this problem. The proposed algorithm produces a summary that is relevant to the user's given query, and it also contains updates information. Experiments were conducted on Text Analysis Conference (TAC) datasets, and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics were considered to assess the model performance. The results obtained by the proposed approach outperform those from the existing approaches in the scientific literature, obtaining average percentage improvements between 12.74% and 55.03% in the ROUGE scores.

KurdSum: A new benchmark dataset for the Kurdish text summarization

Summarizing a text is the process of condensing its content while still maintaining its essential information. With the abundance of digital information available, summarization has become a significant task in various fields, including information retrieval, NLP (Natural Language Processing), and machine learning. This task has been extensively studied in languages such as English and Chinese, but research on Kurdish language summarization is lacking. Therefore, we present the first-ever Kurdish summarization news dataset, KurdSum, which includes over 40,000 texts. We collected news articles from Kurdish websites, preprocessed the data, and manually created a summary for each article. We further assessed the performance of our benchmark dataset on four extractive systems (LEXRANK, TEXTRANK, ORACLE, and LEAD0-3) and three abstractive methods (Pointer-Generator, Sequence-to-Sequence and transformer-abstractive). Our experiments showed that the Pointer-Generator approach yielded superior ROUGE (Recall-Oriented Understudy for Gisting Evaluation) scores compared to other techniques and ORACLE outperformed other extractive methods. Our findings offer a promising direction for the summarization of Kurdish text and can contribute to developing NLP tools for processing the Kurdish language. Likewise, the dataset can serve as a benchmark dataset for Kurdish language summarization and a valuable resource for researchers interested in developing Kurdish summarization models.

Decomposition–based multi-objective differential evolution for extractive multi-document automatic text summarization

The central challenge in Automatic Text Summarization (ATS) involves efficiently generating machine-generated text summaries through optimization algorithms. An ATS is a critical component for systems dealing with textual information processing. However, the current approach faces a significant hurdle due to the computational intensity of the process, particularly when employing complex optimization techniques like swarm intelligence optimization alongside a costly ATS repair operator. While the current approach yields impressive Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics for the generated summary, it struggles with inefficiencies, mainly attributed to the substantial optimization time consumed by the ATS repair operator scheme. In order to address this, a novel solution called Decomposition-based Multi-Objective Differential Evolution (MODE/D) is proposed. It is built upon the foundation of Differential Evolution for Multi-Objective Optimization (DEMO) and the weighted sum method (WS), coupled with an innovative ATS repair operator scheme. Through experimentation on Document Understanding Conferences (DUC) datasets, the novel approach of MODE/D – WS is validated by evaluating the results using ROUGE metrics. The outcomes are twofold: a remarkable reduction in serial execution time and a noteworthy enhancement over existing techniques in the scholarly domain, as evidenced by improved ROUGE-1, ROUGE-2, and ROUGE-L scores.

Abstractive summarization with deep reinforcement learning using semantic similarity rewards

Abstract Abstractive summarization is an approach to document summarization that is not limited to selecting sentences from the document but can generate new sentences as well. We address the two main challenges in abstractive summarization: how to evaluate the performance of a summarization model and what is a good training objective. We first introduce new evaluation measures based on the semantic similarity of the input and corresponding summary. The similarity scores are obtained by the fine-tuned BERTurk model using either the cross-encoder or a bi-encoder architecture. The fine-tuning is done on the Turkish Natural Language Inference and Semantic Textual Similarity benchmark datasets. We show that these measures have better correlations with human evaluations compared to Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores and BERTScore. We then introduce a deep reinforcement learning algorithm that uses the proposed semantic similarity measures as rewards, together with a mixed training objective, in order to generate more natural summaries in terms of human readability. We show that training with a mixed training objective function compared to only the maximum-likelihood objective improves similarity scores.

Exploring the Effectiveness of GPT-3 in Translating Specialized Religious Text from Arabic to English: A Comparative Study with Human Translation

In recent years, Natural Language Processing (NLP) models such as Generative Pre-trained Transformer 3 (GPT-3) have shown remarkable improvements in various language-related tasks, including machine translation. However, most studies that have evaluated the performance of NLP models in translation tasks have focused on general-purpose text, leaving the evaluation of their effectiveness in handling specialized text to be relatively unexplored. Therefore, this study aimed to evaluate the effectiveness of GPT-3 in translating specialized Arabic text to English and compare its performance to human translation. To achieve this goal, the study selected ten chapters from a specialized book written in Arabic, covering topics in specialized religious context. The chapters were translated by a professional human translator and by GPT-3 using its translation Application Programming Interface. The translation performance of GPT-3 to was compared to human translation using qualitative measures, specifically the Direct Assessment method. Additionally, the translations were evaluated using two different evaluation metrics, Bidirectional Encoder Representations from Transformers (BERT) score and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metric, which measure the similarity between the translated text and the reference text.The qualitative results show that GPT produced generally understandable translations but failed to capture nuances and cultural context. On the other hand, the quantitative results of the study showed that GPT-3 was able to achieve a relatively high level of accuracy in translating specialized religious text, with comparable scores to human translations in some cases. Specifically, the BERT score of GPT-3 translations was 0.83. The study also found that the Rouge score failed to fully reflect the capabilities of GPT-3 in translating specialized text.Overall, the findings of this study suggest that GPT-3 has promising potential as a translation tool for specialized religious text, but further research is needed to improve its capabilities and address its limitations.

The observed preprocessing strategies for doing automatic text summarizing

It is challenging for humans to keep up with the rapid creation of digital information due to the explosion of digital information. A written document can be analyzed to extract meaningful information using automatic text summarization. This research proposes 16 different experimental settings in which the model developed by IndoBERT will be applied in order to answer the question of how much of an impact preprocessing has on the quality of summaries produced by automatic text summarization. In order to answer this question, the researchers have devised this study. In this study, we will explicitly talk about preprocessing strategies by conducting tests with different combinations of preprocessing techniques. These techniques include data cleansing, stopwords, stemming, and case folding. After that, the recall-oriented understudy for gisting evaluation (ROUGE) assessment will be used to conduct the measurement of the research results. According to the findings of this research, the optimal level of performance may be accomplished by combining the processes of data cleaning and case folding with scores of 0.78, 0.60, and 0.68 for ROUGE-1, ROUGE-2, and ROUGE-L respectively.

The observed preprocessing strategies for doing automatic text summarizing

It is challenging for humans to keep up with the rapid creation of digital information due to the explosion of digital information. A written document can be analyzed to extract meaningful information using automatic text summarization. This research proposes 16 different experimental settings in which the model developed by IndoBERT will be applied in order to answer the question of how much of an impact preprocessing has on the quality of summaries produced by automatic text summarization. In order to answer this question, the researchers have devised this study. In this study, we will explicitly talk about preprocessing strategies by conducting tests with different combinations of preprocessing techniques. These techniques include data cleansing, stopwords, stemming, and case folding. After that, the recall-oriented understudy for gisting evaluation (ROUGE) assessment will be used to conduct the measurement of the research results. According to the findings of this research, the optimal level of performance may be accomplished by combining the processes of data cleaning and case folding with scores of 0.78, 0.60, and 0.68 for ROUGE-1, ROUGE-2, and ROUGE-L respectively

Feature-based approach and sequential pattern mining to enhance quality of Indonesian automatic text summarization

Indonesian automatic text summarization research is developed rapidly. The quality, especially readability aspect, of text summary can be reached if the meaning of the text can be maintained properly. Therefore, this research aims to enhance the quality of extractive Indonesian automatic text summarization with considering the quality of structured representation of text. This research uses sequential pattern mining (SPM) to produce This research use SPM to produce sequence of words (SoW) as structured text representation using PrefixSpan algorithm. Then, SPM is combined with feature-based approach using sentence scoring method to produce summary. The experiment result using IndoSum dataset shows that even though the combination of SPM and sentence scoring can increase the precision value of recall-oriented understudy for gisting evaluation (ROUGE)-1, ROUGE-2, and ROUGE-L, from 0.68 to 0.76, 0.54 to 0.69, and 0.51 to 0.72. Especially, combination of SPM and Sentence Scoring can enhance precision, recall, and f-measure of ROUGE-L that consider the order of word occurance in measurement. SPM increases ROUGE-L f-measure value of sentence scoring from 0.32 to 0.36. Moreover, combination of sentence scoring and SPM is better than SumBasic that used as feature-based approach in the previous Indonesian text summarization research.

CovSumm: an unsupervised transformer-cum-graph-based hybrid document summarization model for CORD-19.

The number of research articles published on COVID-19 has dramatically increased since the outbreak of the pandemic in November 2019. This absurd rate of productivity in research articles leads to information overload. It has increasingly become urgent for researchers and medical associations to stay up to date on the latest COVID-19 studies. To address information overload in COVID-19 scientific literature, the study presents a novel hybrid model named CovSumm, an unsupervised graph-based hybrid approach for single-document summarization, that is evaluated on the CORD-19 dataset. We have tested the proposed methodology on the scientific papers in the database dated from January 1, 2021 to December 31, 2021, consisting of 840 documents in total. The proposed text summarization is a hybrid of two distinctive extractive approaches (1) GenCompareSum (transformer-based approach) and (2) TextRank (graph-based approach). The sum of scores generated by both methods is used to rank the sentences for generating the summary. On the CORD-19, the recall-oriented understudy for gisting evaluation (ROUGE) score metric is used to compare the performance of the CovSumm model with various state-of-the-art techniques. The proposed method achieved the highest scores of ROUGE-1: 40.14%, ROUGE-2: 13.25%, and ROUGE-L: 36.32%. The proposed hybrid approach shows improved performance on the CORD-19 dataset when compared to existing unsupervised text summarization methods.

Abstractive Text Summarization using Pre-Trained Language Model "Text-to-Text Transfer Transformer (T5)"

Automatic Text Summarization (ATS) is one of the utilizations of technological sophistication in terms of text processing assisting humans in producing a summary or key points of a document in large quantities. We use Indonesian language as objects because there are few resources in NLP research using Indonesian language. This paper utilized PLTMs (Pre-Trained Language Models) from the transformer architecture, namely T5 (Text-to-Text Transfer Transformer) which has been completed previously with a larger dataset. Evaluation in this study was measured through comparison of the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) calculation results between the reference summary and the model summary. The experiments with the pre-trained t5-base model with fine tuning parameters of 220M for the Indonesian news dataset yielded relatively high ROUGE values, namely ROUGE-1 = 0.68, ROUGE-2 = 0.61, and ROUGE-L = 0.65. The evaluation value worked well, but the resulting model has not achieved satisfactory results because in terms of abstraction, the model did not work optimally. We also found several errors in the reference summary in the dataset used.

Extractive Text Summarization Using Deep Learning for Tigrigna Language

With the ever-increasing amounts of textual material such as web pages, news articles, blogs, microblogs, and similar, the Internet became the massive body of unstructured information. In this paper to deal with the issues for the availability of more and more information with less time, the extractive text summarization using the deep learning model was used. In this paper, the proposed approach uses three basic stages of feature extraction, feature enhancement, and summary generation of the given news article to extract the core information, to produce well understandable summary and save reader’s time. In the feature extraction, We explore various features to improve the extracted sentences to the summary by the score and rank of the extracted features matrix by calculating the top thematic words, paragraph segmentation, sentences length & position, proper nouns, and TF-ISF, and the sum of the feature vector given to RBM to enhance the extracted feature vector and finally generate the final summarization by taking top high scores and 50% 0f the sum second higher scores from the enhanced feature extracted scores. For experimenting purpose, we have used 10 news articles from the total gathered news articles gathered from BBC-Tigrigna, Fana-Tigrigna and VOA-Tigrigna news website. The evaluation of the extracted summary was evaluated using Recall-Oriented Understudy for Gisting Evaluation (ROUGE) to compare the system extracted summary with the reference / manual summary prepared by human experts. According the experimentation, the average score of ROUG-1 shows 49% for recall, 39% precession, 42% for F-score and for the ROUGE-2 shows that 32% recall, 26% precession and 28% for F-score, for ROUGE-l also shows that 39% of recall, 33% of Precession, and 35% of F-scores. The result shows the proposed approach have higher result in Rouge-1 and the F-score or harmonic mean of precision and recall is 42% and it solves the problems of information overloading in the ever-increasing available news articles by generating the extractive summarizations.

SMATS: Single and Multi Automatic Text Summarization

In today’s world, the rapid growth of textual data on internet sites & online resources makes it challenging for human being to assimilate essential information. To handle such issues, text summarization (TS) plays an important role. Through the TS process, a shorter version of the original content is generated to preserve the relevant information. This study suggests a quantitative assessment of models for single and multi-document summarization based on the sentence scoring method. Experimentation of the models has been carried out on DUC datasets. A detailed comparative analysis of the models is reported with respect to the performance of algorithms based on various metrics such as Recall Oriented-Understudy for Gisting Evaluation (ROUGE), Range, Co-efficient of Variation (CV) and Readability score.